Randomized Singular Value Decomposition Research Articles

Faster nonconvex low-rank matrix learning for image low-level and high-level vision: A unified framework

This study introduces a unified approach to tackle challenges in both low-level and high-level vision tasks for image processing. The framework integrates faster nonconvex low-rank matrix computations and continuity techniques to yield efficient and high-quality results. In addressing real-world image complexities like noise, variations, and missing data, the framework exploits the intrinsic low-rank structure of the data and incorporates specific residual measurements. The optimization problem for low-rank matrix learning is effectively solved using the nonconvex Proximal Block Coordinate Descent (PBCD) algorithm, resulting in nearly unbiased estimators. Rigorous theoretical analysis ensures both local and global convergence. The PBCD algorithm updates blocks of variables iteratively with closed-form solutions, adeptly handling nonconvexity and promoting faster convergence. Notably, the framework incorporates the randomized singular value decomposition (RSVD) technique and introduces a continuous strategy for adaptive model parameter updates. These strategic choices reduce computational complexity while maintaining result quality. They offer fine-tuned control over the desired rank of the learned matrix and enhance robustness in a straightforward manner. Furthermore, the versatility of the proposed nonconvex PBCD algorithm extends to handling problems with multiple variables, as supported by theoretical analysis. Experimental evaluations, spanning various image low-level and high-level vision tasks such as inpainting, classification, and clustering, validate the effectiveness and efficiency of our framework across diverse databases. The source code is available at https://github.com/ZhangHengMin/FNPBCD_LR.In a nutshell, our framework provides a unified solution to tackle both low-level and high-level vision tasks in images. By combining fast nonconvex low-rank matrix learning with adaptive parameter updates, we achieve efficient computation, yielding high-quality results that demonstrate robustness against various types of noise. The evaluations further endorse the reliability and applicability of our proposed framework.

Fast computation of the eigensystem of genomic similarity matrices

The computation of a similarity measure for genomic data is a standard tool in computational genetics. The principal components of such matrices are routinely used to correct for biases due to confounding by population stratification, for instance in linear regressions. However, the calculation of both a similarity matrix and its singular value decomposition (SVD) are computationally intensive. The contribution of this article is threefold. First, we demonstrate that the calculation of three matrices (called the covariance matrix, the weighted Jaccard matrix, and the genomic relationship matrix) can be reformulated in a unified way which allows for the application of a randomized SVD algorithm, which is faster than the traditional computation. The fast SVD algorithm we present is adapted from an existing randomized SVD algorithm and ensures that all computations are carried out in sparse matrix algebra. The algorithm only assumes that row-wise and column-wise subtraction and multiplication of a vector with a sparse matrix is available, an operation that is efficiently implemented in common sparse matrix packages. An exception is the so-called Jaccard matrix, which does not have a structure applicable for the fast SVD algorithm. Second, an approximate Jaccard matrix is introduced to which the fast SVD computation is applicable. Third, we establish guaranteed theoretical bounds on the accuracy (in L2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_2$$\\end{document} norm and angle) between the principal components of the Jaccard matrix and the ones of our proposed approximation, thus putting the proposed Jaccard approximation on a solid mathematical foundation, and derive the theoretical runtime of our algorithm. We illustrate that the approximation error is low in practice and empirically verify the theoretical runtime scalings on both simulated data and data of the 1000 Genome Project.

Fast and accurate out-of-core PCA framework for large scale biobank data.

Principal component analysis (PCA) is widely used in statistics, machine learning, and genomics for dimensionality reduction and uncovering low-dimensional latent structure. To address the challenges posed by ever-growing data size, fast and memory-efficient PCA methods have gained prominence. In this paper, we propose a novel randomized singular value decomposition (RSVD) algorithm implemented in PCAone, featuring a window-based optimization scheme that enables accelerated convergence while improving the accuracy. Additionally, PCAone incorporates out-of-core and multithreaded implementations for the existing Implicitly Restarted Arnoldi Method (IRAM) and RSVD. Through comprehensive evaluations using multiple large-scale real-world data sets in different fields, we show the advantage of PCAone over existing methods. The new algorithm achieves significantly faster computation time while maintaining accuracy comparable to the slower IRAM method. Notably, our analyses of UK Biobank, comprising around 0.5 million individuals and 6.1 million common single nucleotide polymorphisms, show that PCAone accurately computes the top 40 principal components within 9 h. This analysis effectively captures population structure, signals of selection, structural variants, and low recombination regions, utilizing <20 GB of memory and 20 CPU threads. Furthermore, when applied to single-cell RNA sequencing data featuring 1.3 million cells, PCAone, accurately capturing the top 40 principal components in 49 min. This performance represents a 10-fold improvement over state-of-the-art tools.

Hybrid Nature-Inspired Optimization and Encryption-Based Watermarking for E-Healthcare

With the growth and popularity of the utilization of medical images in smart healthcare, the security of these images using watermarks is one of the most recent research topics. This algorithm is based on the joint use of dual watermarking, nature-inspired optimization, and encryption schemes utilizing redundant-discrete wavelet transform (RDWT) and randomized-singular value decomposition (RSVD). The key idea of the proposed method is to embed system encoded media access control (MAC) address in patient's ID card image via discrete wavelet transform (DWT) to generate the final mark. Afterward, embed the generated watermark into computed tomography (CT) scan images of the COVID-19 patient and general images through employing the RDWT and RSVD. Further, we use a hybrid of particle swarm optimization (PSO) and Firefly optimization techniques to determine the optimal scaling factor for embedding purposes. After that, the watermarked CT scan image is encrypted using an encryption technique based on a nonlinear-chaotic map, random permutation, and singular value decomposition (SVD). Extensive evaluations establish the benefit of our proposed algorithm over the traditional schemes. The optimal robustness is more effective than the five traditional schemes at lower computational efficiency.

Multimodal Fusion-Based Image Hiding Algorithm for Secure Healthcare System

The development of artificial intelligence (AI) plays a significant role of multimedia applications, especially in the healthcare domain. However, it has brought about the problem of sensitive information leakage. To address these challenges, an interesting multimodal fusion-based robust image hiding algorithm is proposed in this paper. Firstly, fused image is considered as mark image generated from MRI and CT images using non-subsampled shearlet transform (NSST). Secondly, we employed principal component analysis (PCA) to compute the appropriate coefficients of cover image for embedding purpose. Thirdly, fused mark image is Arnold cat map encoded to address the security issue hidden mark media. Finally, the fusion of fractional dual-tree complex wavelet transform (Fr-DTCWT) and randomized singular value decomposition (RSVD) is utilized to conceal encrypted fused mark media inside host image. Our findings show that the proposed algorithm outperforms some of the recent techniques in terms of high robustness and invisibility.

Hybrid quantum singular spectrum decomposition for time series analysis

Classical data analysis requires computational efforts that become intractable in the age of Big Data. An essential task in time series analysis is the extraction of physically meaningful information from a noisy time series. One algorithm devised for this very purpose is singular spectrum decomposition (SSD), an adaptive method that allows for the extraction of narrow-banded components from non-stationary and non-linear time series. The main computational bottleneck of this algorithm is the singular value decomposition (SVD). Quantum computing could facilitate a speedup in this domain through superior scaling laws. We propose quantum SSD by assigning the SVD subroutine to a quantum computer. The viability for implementation and performance of this hybrid algorithm on a near term hybrid quantum computer is investigated. In this work, we show that by employing randomized SVD, we can impose a qubit limit on one of the circuits to improve scalibility. Using this, we efficiently perform quantum SSD on simulations of local field potentials recorded in brain tissue, as well as GW150914, the first detected gravitational wave event.

Accelerated matrix completion algorithm using continuation strategy and randomized SVD

The matrix completion (MC) problem aims to recover the complete matrix from the observed matrix with missing entries. The unknown matrix is generally assumed to have a low-rank structure. It is well known that the low-rank minimization problem is NP-hard, so the rank function of the matrix is approximated by a surrogate function. In this paper, we firs t apply the optimization transfer technique to formulate the minimization problem into a bi-variate minimization problem by adding an auxiliary variable. We apply the alternating minimization method to find the minimizer of the bi-variate minimization problem. For the auxiliary variable, the minimum can be formulated as a linear combination of the observation data and the original variable. For the original variable, the minimum is a generalized singular thresholding (GSVT) operator of the auxiliary variable. However, most existing iteration methods for matrix completion problem use fixed thresholding value and also suffer from full singular value decomposition (SVD), which need more computation time for iteration. Thus, they become inefficient for large-scale problems. In order to handle these limitations, we apply the continuation strategy and the randomized singular value decomposition (RSVD) to reduce the computational costs. The threshold value of the continuous strategy is reduced in each iteration until a given minimum thresholding value is reached, while the RSVD only needs to decompose small-scale matrices. Hence, using both strategies simultaneously can effectively reduce the computation time for iteration. Numerical experiments demonstrate that the proposed algorithm can effectively recover the missing entries both in synthetic and real data.

Randomized singular value decomposition for integrative subtype analysis of 'omics data' using non-negative matrix factorization.

Integration of multiple 'omics datasets for differentiating cancer subtypes is a powerful technic that leverages the consistent and complementary information across multi-omics data. Matrix factorization is a common technique used in integrative clustering for identifying latent subtype structure across multi-omics data. High dimensionality of the omics data and long computation time have been common challenges of clustering methods. In order to address the challenges, we propose randomized singular value decomposition (RSVD) for integrative clustering using Non-negative Matrix Factorization: intNMF-rsvd. The method utilizes RSVD to reduce the dimensionality by projecting the data into eigen vector space with user specified lower rank. Then, clustering analysis is carried out by estimating common basis matrix across the projected multi-omics datasets. The performance of the proposed method was assessed using the simulated datasets and compared with six state-of-the-art integrative clustering methods using real-life datasets from The Cancer Genome Atlas Study. intNMF-rsvd was found working efficiently and competitively as compared to standard intNMF and other multi-omics clustering methods. Most importantly, intNMF-rsvd can handle large number of features and significantly reduce the computation time. The identified subtypes can be utilized for further clinical association studies to understand the etiology of the disease.

A Robust Medical Image Watermarking Scheme Based on Nature-Inspired Optimization for Telemedicine Applications

Medical images and patient information are routinely transmitted to a remote radiologist to assist in diagnosis. It is critical in e-healthcare systems to ensure that data are accurately transmitted. Medical images of a person’s body can be used against them in many ways, including by transmitting them. Copyright and intellectual property laws prohibit the unauthorized use of medical images. Digital watermarking is used to prove the authenticity of the medical images before diagnosis. In this paper, we proposed a hybrid watermarking scheme using the Slantlet transform, randomized-singular value decomposition, and optimization techniques inspired by nature (Firefly algorithm). The watermark image is encrypted using the XOR encryption technique. Extensive testing reveals that our innovative approach outperforms the existing methods based on the NC, SSIM, and PSNR. The SSIM and NC values of watermarked image and extracted watermark are close to or equal to 1 at a scaling factor of 0.06, and the PSNR of the proposed scheme lies between 58 dB and 59 dB, which shows the better performance of the scheme.

Dual Watermarking for Security of COVID-19 Patient Record

In recent years, smart healthcare systems have gained popularity due to the ease of sharing e-patient records over the open network. The issue of maintaining the security of these records has attracted many researchers. Thus, robust and dual watermarking based on redundant discrete wavelet transform (RDWT), Hessenberg Decomposition (HD), and randomized singular value decomposition (RSVD) are put forward for CT scan images of COVID-19 patients. To ensure a high level of authentication, multiple watermarks in form of Electronic Patient Record (EPR) text and medical image are embedded in the cover. The EPR is encoded via turbo code to reduce /eliminate the channel noise if any. Further, both imperceptibility and robustness are achieved by a fuzzy inference system, and the marked image is encrypted using a lightweight encryption technique. Moreover, the extracted watermark is denoised using the concept of deep neural network (DNN) to improve its robustness. Experiment results and performance analyses verify the proposed dual watermarking scheme.

Three-Dimensional Seismic Data Reconstruction Based on Fully Connected Tensor Network Decomposition

Rank-reduction approaches assume that seismic data in the frequency-space domain is of low-rank after a specific pre-transformation. The presence of noise or missing traces will increase the rank; therefore, seismic data can be denoised and recovered via rank-reduction techniques. The iterative weighted project onto the convex set (POCS) framework can be used for noise attenuation and data reconstruction simultaneously. Multichannel singular spectrum analysis (MSSA) is a classic 3D seismic data reconstruction algorithm that rearranges the temporal frequency slices of the data with missing traces into a block Hankel matrix, and then uses randomized singular value decomposition (RSVD) to interpolate slices. To further improve the efficiency and precision of 3D seismic data reconstruction, we introduce the fully connected tensor network (FCTN) decomposition over the Hankel tensor of the frequency slices. We show that our novel rank-reduction method estimates fewer parameters than MSSA, yielding more accurate and robust results. Moreover, FCTN decomposes a fourth-order tensor into four factor contractions, which breaks the limitations that traditional tensor decomposition methods such as CANDECOMP/PARAFAC (CP) and Tucker decomposition cannot establish the connections between different factors, and are less effective at characterizing relationships. The newly proposed approach does not require singular value decomposition (SVD), leading to an overall reduction in computational complexity. Synthetic and field examples are used to compare the performance of our method with MSSA, and our numerical results reveal the better performance of the proposed FCTN decomposition method for seismic data with large gaps or a high missing ratio.

Algorithms with randomization-based acceleration strategies for sound source localization by non-synchronous measurements

The Non-Synchronous Measurements is an efficient technique to improve the acoustic image of conventional beamforming for sound source localization. It is unable to quickly reveal the sound sources of the large machinery which requires a large synthetic array with many microphones to capture enough sound pressure data. In this paper, four Non-Synchronous Measurement algorithms with improved computational efficiency are demonstrated. They are implemented by combining the Alternative Direction Method of Multipliers (ADMM) method with different randomization-based acceleration strategies. The new algorithms are developed firstly by (1) using Randomized Singular Value Decomposition (RSVD) to construct the projection basis, and then by (2) migrating the computation platform from the Central Processing Unit to the Graphics Processing Unit for the Cross-Spectral Matrix completion steps, (3) accelerating the soft threshold shrinkage operator with RSVD, or (4) replacing the soft threshold shrinkage with RSVD-enhanced optimal shrinkage in the Cross-Spectral Matrix completion steps. Numerical simulations are carried out under different conditions, including the number of sequential measurement positions, the step length between two sequential measurement positions, the frequency of the sound sources, and the Signal-to-Noise Ratio of the sound pressure signals measured by the microphones. The sound source localization performance of the improved algorithms is assessed by examining their acoustic maps, and the computational performance of these algorithms is compared through their Cross-Spectral Matrix reconstruction errors and time consumption.

Robust fast PMU measurement recovery enhanced by randomized singular value and sequential Tucker decomposition

The development of cyber-physical power systems raises concerns about the data quality issue of phasor measurement units (PMUs). Low signal-to-noise ratios (SNRs) and data losses caused by malicious electromagnetic interference, false data injections, and equipment malfunctioning may jeopardize the data integrity and availability necessary for power system monitoring, protection, and control. To ensure grid resiliency, this paper proposes a robust fast PMU measurement recovery (RFMR) algorithm based on improved singular spectrum analysis (SSA) of Hankel structures. It utilizes single or multiple channels of PMU time-series to restore the problematic phasor measurements with low-SNR noises and data losses. Additionally, the traditional singular value decomposition (SVD) and Tucker decomposition (TD) in RFMR are replaced by randomized SVD (RSVD) and sequential TD (STD) to reduce the computational complexity in single-channel and multi-channel RFMR, respectively. Numerical case studies demonstrate that the proposed algorithm can recover the noise-contaminated measurements with higher accuracy than existing methods, such as matrix/tensor decomposition approaches and robust principal component analysis (RPCA), and effectively complement the missing data with the observed measurements corrupted by low SNRs. Moreover, the latency margins of various power system synchrophasor application scenarios can be satisfied with the reduced computational complexity.

Algorithm 1022: Efficient Algorithms for Computing a Rank-Revealing UTV Factorization on Parallel Computing Architectures

Randomized singular value decomposition (RSVD) is by now a well-established technique for efficiently computing an approximate singular value decomposition of a matrix. Building on the ideas that underpin RSVD, the recently proposed algorithm “randUTV” computes a full factorization of a given matrix that provides low-rank approximations with near-optimal error. Because the bulk of randUTV is cast in terms of communication-efficient operations such as matrix-matrix multiplication and unpivoted QR factorizations, it is faster than competing rank-revealing factorization methods such as column-pivoted QR in most high-performance computational settings. In this article, optimized randUTV implementations are presented for both shared-memory and distributed-memory computing environments. For shared memory, randUTV is redesigned in terms of an algorithm-by-blocks that, together with a runtime task scheduler, eliminates bottlenecks from data synchronization points to achieve acceleration over the standard blocked algorithm based on a purely fork-join approach. The distributed-memory implementation is based on the ScaLAPACK library. The performance of our new codes compares favorably with competing factorizations available on both shared-memory and distributed-memory architectures.

SecDH: Security of COVID-19 images based on data hiding with PCA

Nowadays, image security and copyright protection become challenging, especially after the COVID-19 pandemic. In the paper, we develop SecDH as a medical data hiding scheme, which can guarantee the security and copyright protection of the COVID-19 images. Firstly, the cover image is normalized, which offers high resistance against the geometric attacks. Secondly, the normalized principal component as embedding factor is computed, which are calculated based on principal component analysis (PCA) between cover and mark image. Thirdly, the medical image is invisibly marked with secret mark based on normalized component, redundant discrete wavelet transform (RDWT) and randomized singular value decomposition (RSVD) is introduced. Finally, Arnold cat map scheme employed to ensure the security of the watermarking system. Under the experimental evaluation, our SecDH tool is not only imperceptible, but also has a satisfactory advantage in robustness and security compared with the traditional watermarking schemes.

FastMIE: Faster medical image encryption without compromising security

In the Internet age, medical image security is very important in the field of e-healthcare. Several encryption algorithms have been developed, but they either do not achieve high security or they are computationally overhead for medical images, and thus may not be suitable for e-healthcare applications. In this paper, we develop a medical image encryption algorithm, namely FastMIE, that can provide security at low computational time. To achieve high security, we encrypt the segmented part of the image by using a key generated by redundant-discrete wavelet transform (RDWT) and randomized-singular value decomposition (RSVD) and scrambled segmented image. To reduce the computational cost, we encrypt only the significant part of the original-image instead of considering the whole image. Compared with the existing works, our FastMIE takes much less time in encrypting and has a greater ability of securing the image.

Long-term traffic flow prediction using multivariate SSA forecasting in SDN based networks

Accurate and real-time traffic flow forecasting plays an important role in optimizing traffic routing enabling adaptive and sophisticated applications on the network. Managing and routing enormous traffic flow with dynamic behavior is a highly challenging task. However, arriving at a precise model for traffic forecasting in a short interval of time is not trivial because of the dynamic nature of traffic flow. A novel multivariate time series framework is designed to analyze and forecast the dynamic traffic flow in SDN based networks. The proposed framework adapts the Multivariate Singular Spectrum Analysis (MSSA) forecasting model and incorporates the Randomized Singular Value Decomposition (RSVD) to improve the accuracy of flow prediction. Simulations are conducted to evaluate the effectiveness of the proposed MSSA method. The proposed method predicts the long-term traffic fluctuation from the observed traffic traces. The SDN controller is trained using the traffic traces and future traffic flows are forecasted. The performance evaluation of the proposed method predicts real-time traffic trends accurately with 2.2% MAPE, 9.44 MAE and 13.803 RMSE. The results show that the learning ability of MSSA helps to forecast future network traffic with low prediction errors.

An Efficient Cross-Gradient Joint Inversion Algorithm for Gravity and Magnetic Data Using a Sequential Strategy

The joint inversion methods of gravity and magnetic data have been presented in numerous literatures and implemented in many real applications. However, almost all of these existing methods still face the problem of low computational efficiency, although some targeted improvements have been put forwarded. To further address this issue, this study proposes an efficient algorithm for the joint inversion of gravity and magnetic data. The new algorithm is developed from the idea of structural coupling by using the cross-gradient function and employs a sequential strategy. A new formula for efficient minimization of the structural similarity term is derived innovatively, in which the minimization of the structural coupling term is obtained by a technical combination of an alternating strategy and the randomized singular value decomposition (RSVD) algorithm. Using the proposed new formula, the dimension of the original inverse problem can be reduced greatly. Furthermore, the RSVD algorithm is utilized to perform the independent inversion and determine an optimal regularization parameter. Several comparative tests on a synthetic example show that the proposed efficient method poses great advantage in efficiency compared with the conventional sequential cross-gradient joint inversion (CSCGJI) method. The proposed algorithm is also successfully applied to the real data from a metallic deposit area in Xinjiang province, China.

Orthogonal Features Based EEG Signals Denoising Using Fractional and Compressed One-Dimensional CNN Autoencoder.

This paper presents a fractional one-dimensional convolutional neural network (CNN) autoencoder for denoising the Electroencephalogram (EEG) signals which often get contaminated with noise during the recording process, mostly due to muscle artifacts (MA), introduced by the movement of muscles. The existing EEG denoising methods make use of decomposition, thresholding and filtering techniques. In the proposed approach, EEG signals are first transformed to orthogonal domain using Tchebichef moments before feeding to the proposed architecture. A new hyper-parameter ( α ) is introduced which refers to the fractional order with respect to which gradients are calculated during back-propagation. It is observed that by tuning α , the quality of the restored signal improves significantly. Motivated by the high usage of portable low energy devices which make use of compressed deep learning architectures, the trainable parameters of the proposed architecture are compressed using randomized singular value decomposition (RSVD) algorithm. The experiments are performed on the standard EEG datasets, namely, Mendeley and Bonn. The study shows that the proposed fractional and compressed architecture performs better than existing state-of-the-art signal denoising methods.

A Joint Inversion Algorithm for the Establishment of High-Accuracy 3-D Marine Gravity Field

The 3-D marine gravity field plays an important role in many practical applications, such as matching navigation, object monitoring, and resource exploitation. However, the accuracy of the gravity field generated by most existing methods is relatively low. To solve this problem, this study innovatively puts forward a new method for establishing a high-accuracy 3-D marine gravity field by using a joint inversion algorithm. The presented method incorporates gravity fields observed at different altitudes, especially those observed near the seafloor, because they contain more prominent short-wavelength signals of the sources. The final joint inversion formulas are developed from the standard form of independent inversion of single gravity datasets, so it is conducive to include different weighting matrices. To achieve the efficiency of the presented method, the nonlinear objective function is minimized iteratively using a fast randomized singular value decomposition (RSVD) technique. The employing of RSVD also facilitates the determination of an optimal regularization parameter based on the generalized cross-validation criterion. The presented method is tested on both synthetic gravity fields simulated by a theoretical density model and real gravity fields collected in the South China Sea. Numerical results demonstrate that the presented method can provide more accurate 3-D gravity fields than the commonly used iterative downward continuation method in the wavenumber domain (DCW) and the conventional equivalent source (CES) method.